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ABSTRACT 

Needs assessment identifies '‘real world" problems to 
result in priorities for action. It is used for educational planning, 
but is inconsistently practiced and poorly researched. This paper 
describes how 14 novices conducted needs assessments and explains the 
outcomes. The research context was a year-long fellowship in medical 
education in which the 14 physicians conducted needs assessments in 
preparation for creating a medical curriculum. Twenty findings 
document: (1) activities conducted and sources contacted; (2) 

facilitating and hindering factors; (3) what participants learned 
about the process; and (4) changes to the product (curriculum) 
resulting from the needs assessment. The conceptual framework of C. 
Argyris and D. A. Schon for theories of action was used to derive 
four c one 1 us i ons , which taken together, stress the ways in which 
theoretical and ideal needs assessments differ from "real world" 
needs assessment. The paper suggests methods practitioners across the 
professions can use to conduct needs assessments more effectively and 
efficiently. (Contains 7 tables and 32 references.) (Author/SLD) 
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Needs Assessment: 

What difference does (and can) it make? 

ABSTRACT 

Needs assessment identifies “real world” problems which result in priorities for 
action. It is used for educational planning, but is inconsistently practiced and poorly 
researched. This paper describes how fourteen novices conducted needs assessments and 
explains the outcomes. Twenty findings document: 1) activities conducted and sources 
contacted, 2) facilitating and hindering factors, 3) what participants learned about the 
process, and 4) changes to the product (a curriculum) resulting from the needs 
assessment. Argyris & Schon's conceptual framework for theories of action was used to 
derive four conclusions. The paper suggests methods practitioners across the professions 
can use to conduct needs assessments more effectively and efficiently. 
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Nowhere is the presence offer 'or and the absence of 
prescriptive detail more obvious than in the topic of needs 

assessment. 

(Rossett, 1982, p. 28) 

Whether it is called “needs assessment,” “needs analysis,” “front end analysis,” 
“goal analysis,” “task analysis,” “strategic planning,” or any of many other terms, there 
is general agreement that a systematic approach to problem solving includes an early 
phase in which data are collected and analyzed to identify and describe needs, those 
needs are prioritized, and potential solutions to the prioritized needs are generated. In 
this paper, the term “needs assessment” will be used to describe “any systematic 
approach to setting priorities for future action” (Witkin, 1984, p. ix), with a special 
focus on the use of needs assessment in education. 

Needs assessment can have a far reaching impact. The concepts and techniques 
of needs assessment are applicable in many settings including education, business, 
industr'\ and public and private service agencies (Witkin, 1984; Kaufman & English, 
1979). Also, needs assessment is important because it identifies the "problem" or 
“priority” that effort is to be expended on (Burton &. Merrill, 1991; Geis, 1986; Mager, 
1988). And finally, the parameters (or “values”) used in the needs assessment to 
identify and select the problem also influence the rest of the process (Guba & Lincoln, 
1982; Kaufman, 1977a). As such, needs assessment can enhance or impede the 
likelihood that a solution will yield substantive positive change. 

However, there is a discrepancy between the theoretical use and benefits of needs 
assessment and its real world application. A major preoccupation in the literature is the 
continuing failure of needs assessment to fulfill its promise. It appears that needs 
assessments are often not well done or skipped entirely (Rossett, 1990, 1992; Roth. 1978; 
Wanamaker, 1986). And even when done, the results are rarely used in the later stages of 
the problem solving venture (Benjamin, 1989; Wanamaker, 1986; Witkin, 1984). 

Furthermore, when people attempt to conduct needs assessments, problems 
often arise. The literature posits several reasons for these problems. First, there is no 
commonly held understanding of what needs assessment is. This is evident in debates 
over terminology and components of the process. Terminology describing the process 
differs drastically among authors (Rodriguez, 1988; Rossett, 1986; Sarthory, 1977; 
Sleezer, 1992; Trimbv, 1979). To further confuse the would-be assessor, competing 
models present widely divergent representations of the process. In fact, there are so 
many models for conducting a needs assessment that a portion of the literature is 
devoted exclusively to comparing and contrasting models (Trimbv, 1979; Wanamaker. 
1986; Witkin, 1977, 1984) or presenting decision aids to guide the selection ot an 
appropriate model (Cohen, 1981; Witxin, 1978a, 1978b). 
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A second problem when attempting to conduct a needs assessment relates to the 
significant, yet often unconscious, influence the person conducting the needs 
assessment has upon the process and results. The idea of “need” is vague, and what is 
classified as a need is likely to be highly dependent upon the values of the person 
conducting the needs assessment (English & Kaufman, 1975; Harless, 1985; Guba & 
Lincoln, 1982). Needs assessment can be conceived of as at least two types of decision 
making. Needs assessments result in decisions as to the products : the priority 
problems, causes and solutions. Also, each choice made by the person designing ' n i 
conducting the needs assessment regarding methods to use, types of data to colle _t, 
who to consult, and in what order, can be considered decisions which make up the 
needs assessment proem. The person conducting a needs assessment would be either 
consciously or unconsciously influencing both the process and the final outcome or 
product. 

However, little has been done to close this apparent gulf between the theory and 
practice of needs assessment. Almost all of the literature on needs assessment is 
discursive in nature (Burton & Merrill, 1991). Although these insights are valuable in 
that they are often based on the author’s own experiences and observations, remarkably 
little research has been conducted to substantiate the concerns or sugeest tested ways to 
improve the process. 

The first step in promoting the effectiveness and efficiency of needs assessment 
is to document current practice. Therefore, the purpose of the study reported in this 
paper was to describe how novices 1 conduct needs assessments and explain resultant 
outcomes. Three goals were identified to achieve the purpose of the study. The first 
goal was to develop an understanding of how needs assessment is practiced by 
describing the needs assessment process people use. The process can be understood in 
terms of the types and numbers of activities conducted, as well as the types and 
numbers of sources involved. A second goal was to identify what phenomena facilitate 
and/or hinder people as they perform needs assessments. Needs assessments cannot be 
earned out in isolation. Needs assessors must interact with other people and deal with 
environmental pressures to find valuable data, to analyze information, and, most 
important, to use the results. A third goal was to identify what people learn as a result 
of conducting the needs assessment. "Learning" was defined both in terms of the 
results obtained from the needs assessment (the "products"), as well as what was 
learned about needs assessment itself (the "process"). 



Novices were selected because 1) novices should be better able to describe what they arc doing and 
why, than would experts (Ericsson & Simon, 1984); and, 2) by studying novices’ first attempts at 
needs assessments, more could be learned about how to better prepare people to effectively and 
efficiently conduct needs assessments. 
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METHODS 

The research context was a year long fellowship in medical education, in wh’ch 
primary care physicians from across the US and Puerto Rico developed their skills in 
teaching, research, curriculum development, and administration. The subjects of this 
study were fourteen physicians conducting needs assessments in preparation for 
creating a medical curriculum. 

In this qualitative study, data were collected and analyzed in four phases 
spanning a ten month period. Data included observations, interviews, and documents 
generated by the participants before, during, and after they received training for and 
conducted their first needs assessments. The data collection methods and phases of data 
collection and analysis are detailed in Table 1 “Phases of the study.” Care was taken to 
collect data that sampled across time (ten months), people (14 participants as well as 
people interacting with them), and contexts (formal and informal situations, in the 
training site and while at their home organizations). 

Three levels of data analysis were conducted over the phases of the study. The 
first level of data analysis consisted of repeatedly reading the data set to generate, apply, 
and modify codes. The second level of analysis occurred during phases III and IV and 
consisted of constructing individual case studies from all the data gathered for a 
participant and comparing cases to generate tentative findings. The third level of 
analysis occurred in phase IV. It consisted of a comprehensive review and comparison 
of all data collected in the study according to the constant comparative method of data 
analysis. The third level of analysis closely followed the operations of unitizing, 
categorizing, summarizing, and conducting member checks as described by Lincoln & 

Guba (19S5) and Miles & Huberman (1984). 

Findings were generated and verified by a process of comparative analysis 
which involved categorizing data through convergent and divergent processes. First, 
data were converged into categories describing phenomena relevant to all the data 
contained within the category. Then, in a divergent process, .he data were reviewed 
and each of the categories were refined until they accounted for all the relevant data and 
excluded all other data in other categories. The entire data set for each participant was 
searched for instances of supporting and contradictory evidence prior to attributing a 
discovery to that individual. This means that in tables 3 and 4 even,' item attributed to 
an individual (usually indicated by a check mark) is based on a minimum of one source 
of evidence from observations, interviews and documents with no contradictory 
evidence from these sources. Discoveries for each participant were then compared 
across all fourteen participants. 

Readers are urged to consider the context and participants of the study when 
evaluating the transferability of findings to their own context. It is especially important 
to remember that the study was executed in a training situation in which the participants 
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Table 1 

Phases of the Study: Data Collection and Analysis 



Time 


Purpose 


Interviews 


Observation 


Documents 


Phase 1: 

Before 
Learning 
About NA 
(6 months In 
hom* org., 
,1 1/2 weeks 
on site) j 


PaiSLCQUectipn: 

Enter site and collect 
general data on context 
and background of 
participant (Ps) 


‘Through informal 
inferadior.s 

Emphasis on relationship 
building and developing 
Irusi 


At selected pomis in lime 
during session: 

* Workshops 
’ Lunch periods 
’ After hours 


r Application 

' Site visii /epon 

* Needs Assessment (NA) 
questionnaire io all Ps 

’ Draft idea of major project 
(develop a curriculum) 

’ Copies of some 
workshop exercises 


Data Analysis (First Level): 

Read through early dale to become familiar with Ps and context, and discover ’holes’ where more 
data «s needed 


Phase II: 

While 
Learning 
About NA 
(3 days on 
site) 


f si co'ieciion: 

Ps learning aboul NA 

* Ps designing NA 


* Informal interactions 

Asking elaboration on ps* 
comments and behaviors 
related to NA 


* NA workshop 

(1/2 day) 

* Feedback session in 
which NA workshop is 
evaluated 

* Other limes during 
session 


* Copies of exercises 
generated during NA 
workshop 


Phase HI: 

Doing the 
NA 

(6 weeks in 
homt org.) 


* Examine Phase 1 A 11 data to construct interview guides 


While Ps are in process 
of conducting NA at 
home ir.shlution 


* Phone interview with 
participants 

* Interviews wnh faculty 
who are mentors of Ps 






Data. Analysis (Second L even: 

’ Po^i interviews. begin to construct cases and comoare them to ceneraie tentative hypotheses 


Phase IV: 

Applyir ’ 
NA to 

Curriculum 

(1 week on 
site) 

(12 weeks in 
home org.) 

(1 week on 
site) 

(16 weeks in 
home org. 
and final 1 
week on site 


Daia comedian: I 

On process and 
outcomes of now 
completed NA 


* Informal interactions. 


* Selected po>n:s m time 
during sess-on 

* Tape menior group 
meeting discussing 
needs assessment 
process a^d ouicomes 


* Written NA repoh 

* Annotated B'bhograpry 
assignment 


Through informal 
interaction with Ps and 
faculty to add deiaiis. 
clear up confusions 


' Pnore conversations 
With PS 

‘ Conversations w '.n 
faculty 




* Written commitment on 
major project (lypo of 
curncu'um lo be 
developed) 


Daia Ara'vs s (Second t ovetj: 

Cons*. f uct individual case stories and compare for patterns and Irenes 
* Identify hypotheses to be chocked in final phase of data collection 


Data co’fpciion: 

Seek confirming & 
disaffirming evidence 
of hypotheses 

’ On oe r son?'-ty types 


* Informal mteract-ons 

* Exit interview 


‘ Workshop on 
personality types 

* Selected t<mes during 
session 


* Curriculum document 

* Personalty type 
indicator results 




* Phone converse* ons 

* Individual member 
checks (final week) 

* Presentation ol study 
hypotheses to Ps ;fmal 
week) 


* P's Conference 
presentations 




* Fu‘ ! b'owr' a^aVS'S with memoer checks 



"NA" = Needs Assessment P" = Participants 
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were required to conduct and report upon a needs assessment in a fixed amount of time 
(6 weeks) for the purpose of developing a medical curriculum. It may also be important 
that the participants were novices to needs assessment, working alone to conduct needs 
assessments in their home organisations, and that they were often experts in the subject 
matter area of the curriculum. Study findings should most readily transfer to similar 
contexts and types of needs assessors. 

RESULTS 

Twenty findings summarize the needs assessments of the fourteen study 
participants. These findings are grouped and reported according to the three major 
goals of the study (see Table 2). As space precludes detailed reporting of all findings, 
only selected findings w , be highlighted in this paper. 2 

What did the participants do in conducting a needs assessment? 

Seven findings were generated in answer to this first question. How the 
fourteen participants conducted their needs assessments was largely answered by 
examining the methods they employed as well as the sources they consulted. A 
summary of the needs assessment activities for each participant is presented in Table 3. 
The participants are listed in Table 3 according to the number of activities performed 
with those performing the same number of activities listed according to the degree of 
effort expended on each activity. 3 

Assessors did not have a clear starting and ending point when cond uctinga 
needs assessment (Finding #1 ). The columns of Table 3 labeled prior to and after 
indicate the frequency with which participants conducted needs assessment i eluted 
activities outside of the prescribed six week period for the needs assessment 
assignment. Participants were initiating activities that quality as needs assessment 
before they were formally introduced to the process, and about half continued with 
needs assessment activities after the required assignment was completed. 

Approximately half of the participants used the word "informal" in describing 
what they had done before they had learned of the needs assessment assignment. A 
common trend in all of the early activities was the participants desire to get a handle 
on things." Participants spoke of looking for literature because they "wanted to find out 
what was out there." Those who mentioned talking to colleagues spoke in terms of 
getting input on the topic for the curriculum they would create and/or refinements tor 
the curriculum they already had in mind. As an example ot the first situation. Max said 



2 A detailed description of the entire study can t>c found in Csetc, J. M. (1994) A Qualitative study of 
needs assessment: The cull between theory and practice (Doctoral Dissertation, Michigan State 
University, East Lansing). 

1 Pseudonyms have been used to maintain confidentiality. 
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Table 2 

Study Findings 



#1 : What do the participants do In conducting a needs assessment? 



Finding 1 : 
Finding 2: 

Finding 3: 

Finding 4: 

Finding 5: 

Finding 6: 

Finding 7: 



Needs assessment did not have a clear beginning and ending. 

Taking a block of time to plan was the most effective way to begin the 
needs assessment. 

Assessors used informal and less structured activities more often than 
formal structured activities when conducting needs assessments. 
Assessors did not involve representatives of every stakeholder group in 
the needs assessment. 

Data collection was focused on sources mosi like the assessors 
themselves. 

Data collection was focused on sources within the assessor's immediate 
context. 

The purpose of the contact, and the data collection approach used, was 
dependent upon the type of source being contacted. 



#2. What factors facilitate and/or hinder the needs assessment process? 

The Individual Needs Assessor 

Finding 8: Needs assessments were hindered by the vagueness of the concept for 

beginners. 

Finding 9: Needs assessments were hindered by unexpected time demands and 

delays. 

Finding 10: Needs assessments were facilitated by assessors possessing moderately 

high expectations. 

Finding 1 1 : Needs assessments were facilitated by the presence of key skills in the 

assessor. 

Finding 1 2: Needs assessments were hindered by a too wide or too narrow focus. 

The Training Program 

Finding 13: Needs assessments were facilitated by timely access to helpful materials and 

human resources. 

The Home Organization 

Finding 14: Assessors demonstrating a belief in interna! control perceived fewer 

difficulties in conducting the needs assessment. 

Finding 15: Assessors who were flexible experienced fewer difficulties in conducting the 

needs assessment. 



#3. What do participants learn about the process and products of needs 
assessment? 



Finding 16: 

Finding 17: 

Finding 18: 

Finding 19: 
Finding 20: 



Needs assessment concepts were clearer after assessors experienced the 
process. 

Participants perceived two key benefits from conducting a needs 
assessment. 

Participants clearly defined conditions under which they would periorm 
needs assessments in the future. 

There were few changes to the curricula as a result of the needs assessment. 
Assessors had decided upon the problem and solution before beginning the 
needs assessment. 
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"We’ve talked about this all year, not anything formally, but in talks between ourselves, 
or it has come up in meetings. Now I'm going to have a questionnaire for them to 
answer." As an example of the second, Rubin said, "Talking to people about what I 
wanted to do helped me verbalize my vision. ...every time I told it to someone, it got 
easier." It appears that a lot of informal data collection, and idea generating and testing 
goes on before a person ever begins “doing” the formal needs assessment. 

Table 3 

Needs Assessment Related Activities 



PRIOR TO 


DURING NEEDS ASSESSMENT ASSIGNMENT 


AFT 


ER 














Documents 




Talked 

to 

People 


More 

Readings 




Before 

Training 

Began 


Literature 

Review 

Required 


Meetings/ 

Informal 

Talks 


Contact 

Experts® 


Written 

Survey 


In Context 


External 


Guided 

Interview 


Max 


/ 


/ 


/ 


/ ol 


/ 


/ 


✓ 


/ 




/ 


Joan 


/ 


/ 


/ 


<f (i+e) 


t 


/ 


/ 


/ 


/ 




Mike 


/ 


/ 


/ 




/ 


/ 


/ 


/ 


/ 




Julie 


/ 


/ 


/ 


3 


/ 


/ 


/ 


/ 






Rubin 


/ 


/ 


/ 


/ 




/ 




/ 


/ 


/ 


May 


/ 


/ 


/ 


/ 


/ 






/ 






Emma 




/ 


/ 


/ 


/ 


/ 


/ 








Bill 


/ 


/ 


✓ 


/ (u) 


/ 




/ 




4 


1 4 


Sue 




/ 


/ 


/ 


/ 








/ 




Alexis 


/ 


/ 


/ 


/ (u X'+e) 


/ 












Sam 


2 


/ 




/ 


/5 


/ 


/ 








Eve 




/ 


/ 


/ (u) 


/ 












Rachel 


/ 


/ 


/ 
















Erin 


/ 


/ 


/ 

















Key: 

1 : D'<J prior lo needs assessmen i assignment 

2: "A lot ol the siuf? tor this curriculum was already set." 

3; - l considered contacting some experts, but what they think s already in the literature." 

4: B'H loti the training program snortly after completing the neecs assessment, and could noi conhr.^e wo r k on this cumcu 

5: Sent out surveys bul none were returned 

6: Exports outside own organization unless noted as below 

(i) Contacted experts internal to me organization 
(i+«) Contacted both internal and external experts 
(u): U'ged by fellowship faculty to contact experts 
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Asse ssors used informal and less structured activities more often than formal 
structured activities when conducting needs assessments (Finding #3) . In Table 3, the 
seven major needs assessment activities performed by the participants during the 
assignment period are listed from left to right according to frequency. Each of the 
activities can be categorized according to the approach participants took. Three of the 
seven qualified as structured formal activities because they included advanced 
consideration of the types of information to be sought, methodical collection of data 
from sources, and a documented analysis process. The literature search, written 
surveys, and guided interviews fit this category. The remaining four activities 
conducted during the needs assessment assignment period qualified as informal. For 
the more informal approach, participants did not report written advanced preparation, 
documentation ot the data gathered, or the analysis process. Holding meetings, 
informal talks, contacting experts, and examining documents were activities that fell 
under the informal approach. It should be noted that this difference was not associated 
with the level of formality of the interaction with the respondent-both categories 
included catching people in 'he hallways" as well as calling ahead for appointments. 

There were 30 total instances of structured formal activities among the fourteen 
participants (14 being the required literature review) in contrast to 37 instar '*es of less 
structured informal activities (none of which was required). Setting aside the most 
frequent activity, the formal literature review, the two next most frequent activities were 
informal in nature. In addition, there was considerable variability in the ’"formality” 
with which surveys were designed, conducted and analyzed. Six of the ten participants 
using surveys reported having drafts reviewed and making revisions prior to 
disseminating them. Three other participants openly admitted typing up the survevs "in 
a few minutes without reference to other materials. The review of techniques used 
suggests that, unless specifically required, assessors tend to prefer informal activities 
such as talking with people and perusing documents over more formal activities which 
structure data collection and analysis and require written documentation. 

As sessors tended to base a greater proportion of the needs assessment upon 
particular types of s ourc es (Findings #4 - #7 ). The sources participants consulted 
during the six week assignment period are displayed in Table 4. In the needs 
assessment training session participants generated lists of people likely to be affected by 
the curriculum they' were to create. Participants recognized a variety of groups as 
stakeholders in the intended curriculum, and each ot the stakeholders were contacted as 
sources with the exception of one group: patients (Finding #4). Seven of ten identified 
patients as being affected, but none incorporated them into the needs assessment 
process. This is true even tor more than half of the cases in which students and 
residents would be learning in the curriculum by practicing their skills on patients. 
Findings #5 and #6 suggest a reason: patients were not consulted because the 
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participants perceived patients as being only distantly related and outside the immediate 
context. 

Participants tended to collect data from sources with which they had the most in 
common (Finding #5). Participants saw their peer faculty members, other academic 
physicians, as people that should be involved in the needs assessment. Similarly, their 
learners, people who were soon to become physicians, were also seen as stakeholders 
in all instances. With only one exception, in which current learners were not available, 
all fourteen participants collected information from both peer faculty and learners. 

Table 4 

Sources Consulted 





PEOPLE 


PRINTED MATTER 




In Own Environment 


Externa! 


External 


Inlernat 




Peer 

Faculty 


Current 

Learners 

{Student 

or 

Resident) 


Local 

Superior 


Support 

Staff 


Graduales 


Experts 

m 

Distant 

Peers 

("P") 


Literature 

’'Required 


External 

Documents 


Internal 

Documents 


Mike 


y 


/ 


/ 


/ 


/ 




/ 


/ 


/ 


Joan 


y 


/ 


/ 




/ 


y e 


/ 


/ 


no evals 

/ 


Max 


y 


none 

available 


y 




/ 


✓ EP 


/ 


/ 


/ 


Julie 


y 


/ 


y 




/ 




/ 


/ 


/ 


May 


y 


/ 




/ 


/ 


/E 


/ 




no evals 


Rubin 


y 


/ 


y 






/ E 


/ 




/ 


Bill 


y 


/ 


y 


/ 




yp 


/ 






Emma 


y 


/ 








y e 


/ 


/ 


/ 


Sue 


y 


/ 


y 






yp 


/ 




no evals 


Rachel 


y 


/ 


y 


/ 






/ 




no evals 


Eve 


y 


/ 


y 






yp 


/ 




no evms 


Erin 


y 


/ 




/ 






/ 




3 


Alexis 


y 


/i 




/ 




y e 


/ 




no evals 


Sam 


y 2 


/2 


y 






yp 


/ 


/ 





1: Surveyed populat c^s that were not iriended learners (ethers may have surveyed facul'y. but ni.vays also induced 

intended learners; 

2' Mam activity for these sources was a survey which was never returned 

3 Lcckcd evalua' c^.s prior to beginning NA 

"no eva's’ No evni'jV :n data on learners or curriculum available 

'E*. Contacted *v*»erts” identified Irom literature or (rem relerences 

'P' Contacted *ceer” Incully in similar positions nl other institutions 
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There was progressively less recognition and involvement of people that held 
es less similar to the participants’. Ten of the fourteen participants contacted a local 
superior, another physician of a higher rank in the organization, but this was sometimes 
for political reasons rather than to actually collect needs assessment data. Non- 
physicians in the local environment (nurses, medical assistants, or clerical staff) were 
less often consulted, even in cases where they would have significant involvement in 
making the curriculum work. When interviewed in the midst of conducting his needs 
assessment, one participant stated that he was including support staff almost as an 
afterthought: " A nd last, I'm gonna ask the office staff — nurse practitioners and nursing 
staff- what they think. ..I didn't think of doing that until yesterday, but it occurred to me 
that I should...." Months later, this pan. pant acknowledged that he did not see 
support staff as providing critical input: 



O 

ERLC 



What I really did was take a second pass, hitting some of the people not 
interviewed in the fust pass, because I didn't care what they thought 
then. These were people with no power to stop me, but I did the second 
pass because I wanted to make sure they'd be happy with what I'd come 
up with, before it's totally dropped on them as "this is something you 
have to do." 



When considering stakeholders outside their own organization, participants 
continued to focus on the medical community. When contacting people outside their 
own organization, only one participant contacted a non-physici ,.n. Also, participants 
showed a tendency to contact other members of the medical community with whom 
they had the most in common. As seen in Table 4, five of the ten participants 
contacting people outside their own organization contacted medical faculty in similar 
positions at other medical institutions (marked with a "P," for "peer," rather than "E" 
for "expert" in the published literature). 

Participants demonstrated the same trend of relying on sources most like 
themselves when searching the literature. With one exception, all of the participants 
limited their searches to the medical literature (although several were working on 
psychology or sociology related topics). Participants also displayed a tendency to limit 
their searches of the medical literature information listed within their particular 
specialty. For example, a pediatrician expressed hesitation in looking at guidelines for 
pediatric training published by a family medicine organization (both provide care for 
children). 

In summarizing Finding #5, the participants recognized stakeholders and 
contacted sources that were most like themselves. Most often, peer faculty were 
consulted and collection of data from learners (soon-to-be physicians) was a new idea 
that was accepted by the participants. There was a reluctance to use sources in other 
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specialties of medicine; also, there was almost no contact with sources outside the realm 
of medicine. 

Participants also most often collected data from sources within their immediate 
context (Finding #6). Among the range of people and printed matter sources from 
which participants collected information, six of the sources were contained within the 
participants’ immediate environment, and three were external to it. The balance of the 
effort expended on needs assessment rested with internal sources. There were 55 total 
instances of contacting internal sources among the fourteen participants, in contrast to 
30 instances of consulting external sources (14 being the required literature review). 
Participants often indicated a reluctance to contact external sources, citing difficulty in 
contacting those sources and concerns as to how they would be received. The number 
of external sources contacted may be overrepresented as there was an element ot 
coercion. The training program required a literature review, and staff from the training 
program often urged participants to contact people outside their institution or consult 
external documents, sometimes going so far as to name a person and provide a phone 
number. 

Participants showed a tendency to choose a more or less structured or “formal” 
approach depending upon the source they were contacting (Finding #7). In general, 
formal documented methods were most often used with local sources most like the 
participants. For example, written surveys were used with both peer faculty and 
students by eight of the ten participants that conducted surveys. In contrast, more 
informal and undocumented methods were more often used with sources less familiar 
or similar to the assessors. In four of the six instances in which support staff were 
consulted, informal talks were conducted and there is no written documentation of what 
was asked or said. 

Participants also indicated in interviews and documents that they were not 
looking for the same type of information from all sources (Finding #7). The major 
purposes were to assess support for the project and to collect input on content. 1 o a 
lesser degree, instructional strategies were also solicited. Peer faculty were consulted 
for both support and content. Learners were consulted for support and, to a lesser 
extent, content suggestions. Other sources in positions less like the participants’ were 
contacted for more political rather than data collection purposes: superiors to maintain 
their support for the project, support staff most often (almost as an afterthought) to 
secure their support by making them feel they had some input. The majority of formal 
written data collection and analysis was conducted with data collected from peer faculty, 
learners, and graduates. 
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What factors facilitate or hinder the needs assessment process? 

Eight findings on this second question were derived b - • comparing and 
contrasting the needs assessment experiences of the fourteen participants. The factors 
that facilitated or hindered the needs assessments occurred on the levels of 1) the 
individual needs assessor, 2) the training program, and 3) the home organizations of the 
participants. Consistent patterns were found in which participants who expressed 
frustrations related to needs assessment were thinking and/or behaving differently than 
those who were not expressing as much frustration with the process. Two findings 
related to the individual needs assessor are briefly described below. 

Needs assessments were hindered bv unexpected time demands and delays 
(Finding_#9) . The factors that facilitated or hindered the participants’ needs assessment 
were related to time on task, start time, and participants' time concerns. First, needs 
assessments were, within limits, facilitated by greater time on task. Assessors 
reporting fifty or more hours on their needs assessments used more needs assessment 
methods and contacted a greater variety and number of sources. However, there 
appeared to be a limit to the benefits of increased time on task, as expenditures of close 
to a hundred hours did not necessarily result in more comprehensive needs assessments 
than those in which 50 hours were spent (also supported by findings #1 1 and #15). 
Second, in cases in which needs assessments were started earlier, they were more 
comprehensive. Assessors who delayed the needs assessment spent less time on task, 
conducted less comprehensive needs assessments, and worried more about being able 
to complete their need assessments. Finally, participants reported three different types 
of concerns related to time which hindered their needs assessments. They were: 1) 
surprised by how much ot their own time needs assessment activities required, 2) 
hampered by limitations in other people's time, and 3) were slowed down by time 
delays. 

In t act, in interviews conducted midway through the needs assessment period, 
ten of the fourteen participants described time as the most important and often the only 
problem experienced. Julie's response, when asked what problems or concerns she 
was experiencing in the needs assessment, captured what most of the others also 
expressed: 



Put this in capital letters - TIME.. .it takes so long... I had not anticipated 
needs assessment would be so involved, nor had I anticipated how 
much stan up time would be needed. It took me the better part of a 
WEEK just getting the surveys prepared and formatted so 1 could get 
them out — 1 had absolutely no idea needs assessment would be so 
involved. 
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Many of the participants spoke of being surprised by how much of their own 
time needs assessment took. They had estimated that it would take much less time to 
accomplish each activity. In particular, many of the participants spoke of how long it 
took to create questionnaires, and how much time they had spent on administrative 
tasks such as distributing the questionnaires or setting up interviews. 

NWrk assessments were facilitated bv the p resence of key skills in the assess or 
(Finding #11 ). There was a clear differentiation in particular skill areas between needs 
assessors who described problems and frustrations during the needs assessment 
process and those who did not. Key skills which facilitated the needs assessment 
process were: time management, change gentry, and bias controls in data collection and 
analysis. 4 Assessors demonstrating poorer time management skills experienced more 
difficulty in finding time to work on their needs assessments and in conducting the 
needs assessment in ways that maximized the time available. In particular, time 
management skills made a significant difference in facilitating the needs assessments in 
cases where less time was available. Those skilled in time management nonchalantly 
reported conducting their needs assessments in the midst of their usual daily activities. 

For example, one participant interviewed his learners a few minutes at a time when they 
were "hanging around" in between seeing patients. Two other participants handed their 
surveys out at meetings at which most of the respondents we.e present. 

Needs assessments were facilitated by assessors who recognized the 
relationship between needs assessment and change, and were equipped to employ 
change agentry strategies. For example, one participant described many conflicts with 
others in her organization rising out her needs assessment activities. These conflicts 
could be attributed, at least in pan, to her lack of knowledge and skill in change 
agentryc In what was designed to be a needs assessment meeting with key 
stakeholders, the participant made others aware of the current deficiencies of the system 
they had created, without also devising a graceful way for them to fix the problem 
without embarrassment or too much additional effort. She expressed surprise that 
committee members did not 'jump at the chance to fix the problem." and also said she 
was hurt that "Two of the people that were in that meeting- it seems like they re 
avoiding me now.” In contrast, other participants almost unconsciously incorporated 
change strategies into their needs assessment process, as demonstrated in behaviors 
such as continually talking to people to "keep key people informed" about what is being 

learned over the course of the needs assessment. 

Finally, all of the participants failed to follow one or more common proceduies 
for bias control in data collection and analysis. In some instances, participants later 






J Time management and change agentry skills were ascribed to participants based on consisla aiurn 
in behaviors and statements. The investigator suggests administering instruments measuring linn, 
management and chance aueniry skills 10 participants in future studies. 
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learned that not having followed these procedures hindered the needs assessment 
process. And in many more cases, participants assumed the correctness of their 
conclusions, when experts would question their conclusions based upon the faulty data 
collection and analysis process. Instances in which bias was not controlled were most 
evident in the surveys, because this method was documented. Common bias control 
problems were 1) asking leading questions of respondents, 2) not reviewing or piloting 
data collection instruments, and 3) making assumptions based upon low response rates 

in surveys. In addition, needs assessment conclusions were usually reported without 
providing supporting data. 

What did participants learn about the process and products of needs 
assessment? 

What participants learned about needs assessment was answered by looking at: 
1) what participants learned about the process of needs assessment, including the ~ 
circumstances under which they would voluntarily elect to conduct needs assessment 
again; and 2) the products of needs assessment, as expressed by changes to the 
participants' curricula as a result of having conducted the needs assessment. Four of 
the five findings generated in response to this question are briefly described below. 

P articipants perceptions of t he b enefits of needs assessment Participants 
learned that contacting a variety of sources during the needs assessment were useful for 
1) collecting ideas and 2) gaining support (Finding #17). They perceived the ideas they 
collected from these sources as making their curricula better. Also, they discovered that 
by asking people for ideas, they gained support, making their curricula more likely to 
succeed. In addition, it appears that early collection of ideas from a variety of sources 
simplifies the later stages of development and implementation. 

Second, participants clearly defined conditions under which they would perform 
subsequent needs assessments (Finding #18). All the participants said they would 
conduct needs assessments in the future. However, they defined conditions which 
limited the circumstances under which they would choose to do one. They said they 
would do needs assessments in situations in which they needed to establish or assess 
support for an idea, and in areas that were new to them. Participants felt that needs 
assessments were less useful in areas in which they already had expertise. They further 
indicated that in the future they would not expend as much time and energy on defining 

the problem and identifying a solution. Instead, they would use the needs assessment" 
to determine how to implement their "best guess." 

Th e . etfect of needs assessment upo n the final product There were few 
substantial changes to the curricula as a result of needs assessment (Finding #19). 
Changes related to topic, content and instrucional format were minimal. An 
examination of documents generated before and after the needs assessments were 
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conducted revealed that eleven of fourteen participants did not change their curriculum 
topic as a result of the needs assessment. In two of the three instances in which 
“change” was reported, the participants had used the needs assessment to collect input 
on how to narrow the topic. In one instance a participant repeated discovering during 
the needs assessment that there was a much more pressing need for a different 
curriculum. 

Needs assessment showed slightly more promising results in the area of 
selection of content and instructional methods. Ten of the fourteen participants gave 
evidence of using the needs assessment to help them select content for the intended 
curriculum. One talked about the needs assessment “moulding” her content, and 
another said that he had discovered a “new philosophy” that guided content selection. 
One participant’s comment illustrates how the needs assessment usually led to relatively 
minor changes in content [emphasis added]: 

I definitely had some clear ideas about what I thought should be 
included. But when 1 talked to people, a couple of residents or the 
graduates clearly said 'no' or emphasized other things. They definitely 
emphasized some other things. And that made me feel good. It was like 
I was truly being open minded, that I could incorporate what they were 
telling me. But there weren't many major shifts. It was more 
like small but significant stuff. 

Eight of the fourteen participants used the needs assessment to help them select or refine 
ideas on how to teach and/or evaluate the curriculum. As had happened with content, in 
most of these instances the needs assessment led to minor modifications rather than new 
ideas. 

And finally, it appears that assessors had decided upon the problem and solution 
before beginning the needs assessment (Finding #20). The needs assessment did not 
find a solution. Instead, the solution had been determined prior to the needs 
assessment. A close examination of participants’ written statements oi intention both 
before and after the needs assessment reveals that: 1) there was only one instance of a 
topic changing between before and after the needs assessment, and 2) the majority ol 
participants had selected a topical area, from which they did not deviate, months before 
they entered the training program and began the needs assessment. Returning to what 
was reported in the first finding, participants had done what could be considered an 
"informal needs assessment" to decide upon which problem to work and its solution 
prior to entering the training program. Rather than resulting in changes, the needs 
assessment confirmed and refined these decisions. As one participant said: 
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I'll be honest with yon, [the needs assessment report is] just a 
document. I already had a good idea of what I needed. There are only a 
few areas where I have found things that were not expected. 



In reviewing the results of the study, findings for the 14 participants generally 
support the concerns expressed in the literature over needs assessment being poorly 
understood and unevenly practiced. The following discussion presents a conceptual 
framework for interpreting the findings, the four major conclusions of the study, and 
general recommendations for improving the effectiveness of needs assessment. 



DISCUSSION 



The conceptual framework 

A conceptual framework is useful for organizing a study's findings into a 
comprehensible structure. In the present study, the conceptual framework was taken 
from the work of Argyris & Schon. In Theory into Practice (1974), Argyris and Schon 
address the practice and education of professionals. They have studied the 
interpersonal interactions of professionals such as physicians, social workers, and city 
planners to develop a theory of how people can become more competent and effective. 
Argyris and Schon describe a conceptual framework in which humans behave 
according to theories of action, which are divided into espoused theories and theories- 
in-use. They' further describe a process by which these theories are maintained or 
modified through single-loop and double-loop learning and present two models for 
human interactions. 

The most important features c .f this conceptual framework to the present study 
are the following: 



1) What people think they do (espoused theory) can differ markedly from 
what they actually do (theory-in-use) without their being aware of the 
conflict. 

2) People's behavior can be described by a model (theorv-in-use) which 
includes a system of constructs (governing variables and action 
strategies) to bring about the consequences they desire (maintaining 
governing variables). 

3) There are at least two kinds of learning and changing: one which 
maintains and refines the status quo (single-loop) and one which 
establishes new norms (double-loop). Both types of learning are 
necessary. Single-loop learning is useful for day-to-day adjustments: 
double-loop learning results in more fundamental changes and is 
essential for maintaining long-term effectiveness. 
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4) People's assumptions and behaviors may be predicted and explained by 
models (theories-in-use). There are two different models. In Model I 
people seek to control others without being influenced in turn, resulting 
in a highly defensive situation with decreased effectiveness and little 
opportunity for learning. In Model I little change occurs until the 
situation reaches crisis proportions and an enormous change, often of 
revolutionary proportions, results. Model II, which is largely 
theoretical, increases capacity for learning and effectiveness. It allows 
for the public testing of assumptions with honest feedback so that 
double-loop learning can occur. 

Argyris and Schon's conceptual framework provides a more accurate understanding of 
reasons for current needs assessment practices and suggests how to design more 
effective needs assessment practices. 



Study Conclusions 

The major conclusions of the study were derived by interpreting the twenty 
findings according to the conceptual framework. The conclusions for each of the 
research questions are presented in Table 5. 



Table 5 

Study Research Questions and Conclusions 



Research Question 


Conclusion 


1 . What do the participants do in 
conducting a needs assessment? 


1 . The needs assessment procedures 

prescribed in the literature are different 
from how they were actually carried out 
in the real world. 


2. What factors facilitate and/or hinder the 
needs assessment process? 


2 . The literature and formal training of 
needs assessment concentrated on the 
ideal, rather than a comprehensive set of 
practical procedures that could be used 
to cope with constraints in real settings. 


3. What do participants learn in doing a 
needs assessment? 

a. about Hie process 

b. about lire products 


3a. The benefits of needs assessment 

perceived by the participants differed 
from those described in the literature. 

3b. The results of the needs assessments 
were not as far reaching and change 
stimulating as suggested in the 
literature. 
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Four arguments support and explain the conclusions, drawing from the findings 
and the conceptual framework. The arguments are: 

One: An espoused theory of needs assessment is described in the literature, and 

by the training situation and participants. 

Two: Needs assessment as practiced by the participants is a theory-in-use. 

Three: Differences between the espoused theory and theory-in-use of needs 
assessment create conflict. 

Four: There may be a way to bring the espoused theory and theory-in-use of needs 
assessment into agreement. 

Each argument will be presented in turn. 

Argument One: An espoused theory of needs assessment is described in the 
literature, and by the training program and participants . According to the criteria 
presented by Argyris and Schon, the goals, models and techniques for conducting need 
assessment as described in the literature represent an espoused theory of needs 
assessment. It fits the criteria by being a set of rules people state about needs 
assessment that is not based on observation or verified by empirical research. Two of 
the tenets of the espoused theory of needs assessment are: 1) needs assessment is 
conducted in order to arrive at the priority problem and an appropriate solution; and 2) 
a needs assessment is to be conducted with the intent of better serving society as a 
whole (‘‘Alpha needs assessment"). These tenets are to be accomplished through the 
strategies presented in the literature. Example strategies include: (most often) using a 
discrepancy approach to identify needs, distinguishing needs from wants, collecting 
data from a variety of stakeholders, and so forth. 

The training situation presented an espoused theory of needs assessment in 
formal sessions and materials which was closely related to the theory espoused in the 
literature. Examples include claims of the benefits and importance of needs assessment. 
The espoused theory was evident in training staff’s public references to needs 
assessment as an important early step in a systematic curriculum development process, 
and in the needs assessment workshop handouts. 

Participants also possessed an espoused theory of needs assessment. This 
espoused theory was evident in formal documents, public presentations and in 
statements in interviews. In most cases the participants stated an espoused theory' that 
closely matched that described in the literature. As an example of what their espoused 
theory' included, participants described how it was important to contact a variety of 
sources in their needs assessments (Finding #18). 

Needs assessment, as espoused in the literature, the training situation, and by 
the participants, approximates Argyris and Schon s Model II. As in the first governing 
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variable in Model II, the intent of needs assessment is to collect valid information. This 
is accomplished through involving stakeholders in the data collection and prioritizing 
process. Similar to the second governing variable in Model II, needs assessment is 
designed to involve others on the basis of their own free and informed choice. This 
leads to the third governing variable in Model II. Needs assessment, as espoused, 
contains the belief that by being involved in the process, stakeholders will be committed 
to the decisions that come out of the needs assessment; they will monitor the solution's 
progress, as they have a vested interest in its successful implementation. 

The espoused theory of needs assessment is aimed at achieving double loop 
learning. The three governing variables of Model II lead to double loop learning 
(fundamental changes in the existing norms). In the literature, needs assessment is 
described as the initial stage of any planning effort which lays the groundwork for 
effecting lasting and positive change. It is to be used as a planning tool by questioning 
the status quo and proactively designing new systems and strategies that will bring 
substantial improvements. 

Argument Two: Needs assessment as practiced by the participants is a theory_- 
in-use . The theory of needs assessment, as espoused in the literature, the training 
situation, and by study participants, was not in agreement with the theory-in-use 
employed by the participants when they actually conducted needs assessments. The 
theory-in-use for needs assessment operates according to a different model, contains 
different governing variables, and results in a different outcome for learning and 
change. 

The theory-in-use for needs assessments more closely ascribes to the governing 
variables of Argyris and Schon’s Model I than Model II. This is illustrated by findings 
that lead to each of the governing variables. The first governing variable of Model I is 
"Define goals and try to achieve them." This governing variable assumes individuals 
will construct their own goals and avoid being influenced by others. Several of the 
findings support this governing variable. Finding #20 describes how the participants 
had already decided upon the priority problem and the solution prior to beginning the 
needs assessment required by the training program. Finding #19 demonstrates there 
were few changes to the final product as a result of the needs assessment. Thus, the 
needs assessment substantiated the participants' own original ideas. 

A combination of findings made it unlikely that the participants would be 
confronted with data that would contradict their own ideas. In findings #3 through #7, 
descriptions of methods used and sources contacted by the participants in the needs 
assessment were presented. These findings showed that assessors collected data from 
sources most like themselves (#5) and completely ignored a particular stakeholder 
group (#4). Assessors also focused data collection on sources within their own 
immediate environment (#6). By limiting their exposure to sources in different roles 
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that might have a different perception of the problem, or in different contexts that might 
have alternate approaches to solving the problem, the assessors were less likely to hear 
opinions that were very different from their own. A further layer of protection was 
added through differences in data collection methods. As described in findings # 3 and 
#7, assessors used structured methods that documented responses with sources that 
were most like the assessors and/or within the assessors' own contexts-they used less 
structured, undocumented methods with sources less like themselves and from other 
contexts. Thus, when assessors analyzed the data, they were less likely to be 
confronted with written data that w-ould conflict with their own ideas. 

The second governing variable in Model I is "Maximize winning and minimize 
losing." Argyris and Schon said that "[participants felt that once they had decided on 
their goals, changing them v.ould be a sign of weakness" (1974, pp. 66-67). Thus 
"winning" consisted of ensuring that their original goals would be accepted. In the 
present study, participants employed behaviors that made it more likely that they would 
"win." The findings listed under the first governing variable illustrate how the 
assessors ultimately "won" in the sense that the needs assessment validated their 
preferred solution. The findings also suggest that the needs assessment was conducted 
in more subtle ways that favored the assessor. An example is the lack of bias controls 
described in Finding #11. By asking leading questions and accepting a low response 
rate on questionnaires, participants were biasing data in their favor. Also, by not 
presenting hard data in the needs assessment reports, participants were not allowing 
others to question how their conclusions had been determined. 

The third and fourth governing variables of Model 1 are "Minimize generating or 
expressing negative feelings" and "Be rational." These governing variables assume it is 
better to avoid areas of possible conflict than to uncover painful emotions, even if they 
are closely associated with accurate and pertinent information. These variables require 
assessors to hold back their own feelings and avoid areas in which others may have 
strong feelings. In the present study, participants gave many examples of behaviors 
that could be accounted for by these governing variables. These behaviors were often 
related to the "political" purposes of needs assessment. For example. Findings #5 & 

#1 1 include descriptions of how participants collected data they never intended to use 
for the sole purpose of making sure these sources w'ould not be hurt (and, going back 
to the second variable, so they wouldn't oppose the participants' intended solution). 

This contrast between the espoused theory and theory-in-use of needs 
assessment results in Conclusion 1 : The needs assessment procedures prescribed in the 
literature are different from how they were actually carried out in the real world. The 
difference between the major tenets of the espoused theory' and those of the theory-in- 
use of needs assessment are summarized in Table 6. 
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Table 6 

Sample Comparison of Espoused Theory and Theory-in-use 

of Needs Assessment 



Theory Espoused in Literature and 
by Training Situation and 
Participants 


Theory-in-use Displayed in 
Behavior of Participants 


Needs assessment is conducted in order to 
arrive at the priority problem and an 
appropriate solution. 


Participants had already decided upon the 
priority problem and the solution prior to 
beginning the prescribed needs assessment.. 
(Finding #20) 

The prcsci ibed needs assessment did not 
result in significant changes to the assessor's 
original problem and solution. (Finding 
#19) 

The needs assessment was conducted in a 
way to avoid the assessor being confronted 
witli contrary data. (Findings #3, 4,5, 6, 7) 


A needs assessment is to be conducted with 
the intent of better serving society as a whole 
(Alpha needs assessment). 


Tlic needs assessment substantiated the 
assessor's personal idea. (Findings #19, 20) 

The needs assessment was focused almost 
exclusively within the immediate context 
and upon sources similar to the assessor. 
(Findings #3,4,5 .6,7) 



Argyris and Schon's conceptual framework describes how fundamental changes 
that are necessary to long run effectiveness grow out of changes in the governing 
variables and existing norms. They call these changes "double loop learning." The 
findings suggest the participants' needs assessments resulted in single loop rather than 
double loop learning. Assessors in this study began with a particular problem in mind 
and had a preferred solution. Assessors then conducted their needs assessments in 
ways that were less likely to reveal other problems or lead to new or unusual solutions. 
Assessor choices as to how data was collected, the sources from which data was 
collected, and the ways the data was analyzed made it likely that only small 
adjustments, also known as single loop learning, would occur. 

In summarizing Argument Two, participants practiced a theory-in-use that was 
different from their espoused theory of needs assessment (leading to Conclusion 1 ). 
Their theory-in-use approximated the governing variables present in Argyris t .id 
Schon's Model I, and was likely to result only in minor modification to the status quo, 
or single loop learning (leading to Conclusion 3b). Unfortunately, the disparity 
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between the espoused theory and theory-in-use of needs assessment creates conflicts 
and inhibits effectiveness. This concern is discussed in Arguments Three and Four to 
explain Conclusions 2 and 3a. 

Argument Three: Differences between the espoused theory and theory-in-use of 
needs assessment create conflict . Argyris and Schon assert that inconsistencies 
between espoused theory and theory-in-use lead to conflict. People are not aware of, 
and consequently not able to detect, discrepancies between what they think they are 
doing and their actual behavior. Therefore, when intended outcomes do not occur, they 
are unable to accurately pinpoint the problem and cannot make improvements. By 
applying the conceptual framework to the study Findings, the conflicts between the 
espoused theory and the theory-in-use of needs assessment are made apparent and can 
be explained. 

Argyris and Schon describe how the Model I theory-in-use acts as a self 
fulfilling prophesy, or is "self- sealing." As long as people behave according to the 
governing variables in the Model I theory-in-use, they will continue to confirm their 
own ideas and maintain Model I governing variables. Similarly, participants in the 
present study indicated a reluctance to change their existing behaviors and beliefs. 

Even when participants were made aware of the conflict between their espoused 
theory and theory-in-use of needs assessment, they did not address the conflict. A 
particularly salient example occurred the last two days the investigator was in contact 
with all the participants. As described in Finding #19. the investigator showed 
individual participants tables of data documenting the lack of changes to curricula. 
Participants were visibly surprised when shown concrete examples of how their 
projects had changed little as a result of the needs assessment. However, in these 
private conversations, after expressing surprise, the participants acknowledged that 
there had been no change in their topic and, at most, minor adjustments to content and 
instructional methods were made as a result of performing the needs assessment. 
Nevertheless, in the same two-day period, participants also presented their curricula in a 
public forum and made claims about needs assessment that contradicted this discovery. 
The participants presented their curricula in ten minute presentations at a conference. 
Thirteen of the fourteen participants used a portion of their ten minutes to talk about 
needs assessment. In each of these thirteen instances, participants described their needs 
assessment process and said that it had helped them discover the problem and design an 
appropriate solution. In other words, participants acknowledged their theory-in-use of 
needs assessment in private discussions, but presented the differing idealized espoused 
theory of needs assessment publicly. 

The conceptual framework suggests that ignoring this disparity was expedient 
and, in the short term, useful for the participants. As described in Argument Two, they 
had conducted needs assessment according to their theory-in-use, which had allowed 
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them to confirm their original ideas. Yet, participants appealed to the espoused theory 
of needs assessment to give their needs assessment results and, by extension, their 
curricula, legitimacy. 

Another example of how participants' theory-in-use of needs assessment is self- 
sealing was illustrated in Finding #1 8. When describing the conditions under which 
they would conduct needs assessments again, the participants who had conducted the 
most comprehensive needs assessments stipulated that, in the future, they would not 
use as many data collection methods, contact as many sources, or formally analyze and 
write up the data. By further restricting the data collection, analysis methods, and the 
range of sources contacted in future needs assessments, participants would be even less 
likely to discover problems or solutions that differed from their own ideas. 

The problems with the practice of needs assessment described in the literature 
can be attributed to conflicts between espoused theory and theory-in-use. Problems 
described in the literature include 1) focusing on needs of the individual or organization 
rather than on what would benefit society as a whole (Roth, 1978; Witkin, 1984; 
Kaufman, 1977a, 1977b, 1989), and 2) failure of the assessor to define "need" (Roth, 
1977,1978; Sleezer, 1992). The conceptual framework suggests an explanation for 
why these problems exist. These "problem" areas are actually points at which the 
espoused theory and theory-in-use of needs assessment conflict. 

It is even possible that some of these "problems" continue to exist because 
resolving them would highlight conflicts between the espoused theory' and theory-in- 
use of needs assessment and force changes. For example, conducting needs 
assessment according to a clear operational definition of "need" would make the present 
theory-in-use impracticable. A clear definition of "need" would include whose needs 
were being considered and the criteria by which needs were to be prioritized. This 
would necessitate the inclusion of stakeholders regardless of whether or not that 
population's opinions were likely to be in agreement with the assessor's. Measuring 
need according to a particular approach, discrepancy based for example, would mean 
that other needs might be prioritized above the need on which the assessor prefers to 
work. Thus, conducting a needs assessment according to a clear definition of need 
would make it difficult for assessors to substantiate their own ideas. However, as long 
as the disparity between the espoused theory and theory-in-use continues, assessors 
will be able to use the needs assessment to validate their preferred solution, yet still 
appeal to the espoused theory to claim higher ideals. 

Also, the conceptual framework provides an explanation of why there are 
disagreements over terminology and so many divergent models of needs assessment 
exist. The multitude of terms and models may be endemic to this conflict between 
espoused theory and theory- in -use. For example, Kaufman's (1985) insistence over 
the difference between "needs assessment" (which he describes as problem 
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identification and prioritization) and "needs analysis" (which he describes as the search 
for causes and solutions), can be seen as his attempt to distinguish between espoused 
theory (needs assessment identifies problems) and theory-in-use (needs analysis begins 
with a given problem). 

This conflict between espoused theory and theory-in-use limits the effectiveness 
of needs assessment. Although, in the short term, it is "easier" to ignore conflicts 
between espoused theory and theory-in-use, effectiveness is greatly diminished in the 
long term. One may ask, if needs assessment, as currently practiced, results in few 
changes to the original idea, is it any wonder that needs assessment is often skipped? 

On some level, didn't each of the assessors know that the needs assessment had been 
used to legitimize or refine rather than discover the problem and solution? This conflict 
appears to have 1) complicated the teaching of needs assessment, 2) made it more 
difficult to conduct a needs assessment (especially as a novice), and 3) brought needs 
assessment results into question. 

The limited effectiveness of needs assessment is evident in Conclusion 3a, 
which states that the benefits of needs assessment perceived by the participants differed 
from those described in the literature. Although participants espoused benefits of needs 
assessment in Finding #17 that more closely matched the theory espoused in the 
literature (needs assessments are useful for collecting ideas and gaining support), in 
Finding #18, the participants' contrasting theory-in-use for the benefits of needs 
assessment was evident. Participants stressed the usefulness of needs assessment for 
gaining support far over collecting ideas. They did not perceive problem finding or 
solution identification to be a benefit of needs assessment, except in situations in which 
they were unfamiliar with the content. And even in cases in which they would choose 
to do a needs assessment in the future, their desire to simplify the process by using 
fewer methods and contacting fewer sources suggests they did not perceive that the 
benefits of needs assessment justify much effort. 

Ultimately, failure to resolve these conflicts substantially weakens needs 
assessment. As described in Finding #20, participants had a preferred problem and 
solution in mind, which changed little over the course of the needs assessment. As 
stated in Conclusion 3b: the results of the needs assessments in the present study were 
not as far reaching and change stimulating as suggested in the literature. The 
participants who expressed the most disappointment with the outcomes of needs 
assessment were those who had put the most time and energy into the process. They 
were frustrated when the process did not deliver what the espoused theory had 
promised. Max, w ho spent the greatest amount of time on his needs assessment, spoke 
openly with the investigator about his disappointment. Max felt that the benefits had 
not justified the energy he had invested in the needs assessment. The field notes 
capture what transpired later in the same conversation: 
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Max leaned forward, and gestured, rhythmically pointing his index 
finger to the table as he spoke the following with emphasis. Like tor 
me, before I order any [medical] test, I ask myself, Could I learn 
something from this that would make me change niv mind. And it my 
answer is 'No,' I don't order it." 

I said, "That sounds like the same philosophy as you had about needs 
assessment. It isn’t that useful to you because you didn t change much 
because of it." 

Max: "Yes. Exactly." 



In summary', a case is ade in Argument Three for how the C' tsting theory-m- 
use of needs assessment is self-sealing. Evidence shows thai assessors find it 
expedient to overlook the existing conflict between espoused theory and theory-in-use. 
This conflict may also account for many of the problems with needs assessment as cited 
in the literature and complicate the teaching of needs assessment. Ultimately, this 
conflict greatly decreases the effectiveness of needs assessment. The fourth and final 
armunent will suggest how espoused theory' and theory-in-use may be reconciled. 

Arnnn^m Four There mav be a wav to b ring the esnou s ed theory and theorw 
in-nse of needs assessment into agreement . The espoused theory ot needs assessment is 
well documented in the literature. This study has made a start in documenting the 
theory-in-use of needs assessment. Recommendations from Argyris and Schon s 
experience may provide the next steps in working to bring the espoused theory' and 
theory-in-use of needs assessment into agreement so that the effectiveness ot needs 
assessment can he enhanced. 

Argyris and Schon (1974) propose how to redesign professional education. 
Their suggestions include clarifying the areas of crisis by documenting the conflicts 
between espoused theory and theory-in-use. and working on transforming theories in- 
usc to more closely approximate Model II. How needs assessment is taught and 
practiced may be reformed by Argyris and Schbn's recommendations on how to make 
transitions between the two models. They assert that two key factors in making this 
transition are 1 ) repeatedly examining the theory-in-use through publicly testing 
assumptions and 2) being open to changing behavior. 

However, we should not automatically assume that all the changes must be 
made in the theory-in-use of needs assessment. It is also possible that adjustments 
should he made to the espoused theory to make it more practicable in real world 
settings. For example, needs assessment, as currently espoused, describes a process 
for problem and solution finding. It completely ignores that, at least in the cases 
documented in the present study, assessors do nest approach the task as a tabula rasi. 
Similar to Simon’s ( description of administratis e deewon making, before 
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beginning the prescribed needs assessment, they thought about the situation, perhaps 
conducted some exploratory questioning, and began to employ their own assumptions. 
An espoused theory' of needs assessment needs to take these aspects of human nature 
into account. In considering the prior knowledge and decision making of the assessor, 
the needs assessment process might, under certain circumstances, be better used for 
problem and solution refining. 

In addition, the present study's findings on helping and hindering factors 
suggest other barriers to successfully conducting a needs assessment in actual 
situations. Argyrtsand Schon acknowledge that Model II is difficult to implement in 
existing settings. They contend that "organizations tend to create learning systems that 
inhibit double-loop learning that calls into question their norms, objectives, and basic 
policies (1978, p.4). It would be especially difficult for unprepared assessors to deal 
with this resistance. Issues such as perception of self efficacy, the fact that change 
implied by needs assessment may be threatening to others, and that the assessor may 
have little personal power in the organization must be addressed in an espoused theory. 
The findings related to the second research question may be used to design future 
instruction on needs assessment that prevents or ameliorates some of the hindering 
factors, and begins to create guidelines for periodically checking for conflicts between 
espoused theory and theorv-in-use. 

S ummary of conc l usions an d arguments . In Arguments One and Two the 
differences between the espoused theory and theory-in-use for needs assessment were 
described. These arguments support Conclusions 1 and 3b. which state that the needs 
assessment procedures prescribed in the literature (the espoused theory) are different 
from how they were actually carried out in the real world (the theory-in-use of needs 
assessment), and that the needs assessments carried out by the participants of the 
present study did not accomplish what is claimed in the literature. As described in 
Argument Three, ( '^elusion 2 states, that the espoused theory of needs assessment 
presented in the literature, by the fellowship, and bv participants focuses on the ideal; 
whereas the theory-in-use of needs assessment, as observed in participants' behaviors, 
is, by necessity, much more practical. Evidence was also presented in Argument Three 
to show how this conflict is detrimental. Not only did the participants perceive fewer 
and less important benefits from conducting a needs assessment (Conclusion 3a). but 
the results of the needs assessments in the present study were not as far reaching and 
change stimulating as suggested in the literature (Conclusion 3b). The fourth and final 

argument suggested how espoused theory and theory-in-use could be brought into 
agreement. 
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General recommendations for assessors and those training them 

As described in Conclusion 2, needs assessment, as currently taught, does not 
adequately prepare the assessor to effectively and efficiently perform needs 
assessments. According to the conceptual framework, this is because the espoused 
theory of needs assessment is what is taught, and assessors are not presented with 
strategies to become aware of their theory-in-use of needs assessment and to monitor 
the two theories for conflict. The conflict is essentially between the ideal, as captured in 
the espoused theory, and the practical, as demonstrated in the theory-in-use of needs 
assessment. Both viewpoints must be considered if the effectiveness of needs 
assessment is to be improved. 

Table 7 is a compilation of suggestions from the study findings which 
demonstrate how we can better prepare practitioners to effectively and efficiently 
conduct needs assessments on their own. The general recommendations listed in Table 
7 address two central themes in an effort to bring the espoused theory' and theory-in-use 
of needs assessment into agreement. First, needs assessment must be designed and 
taught in a way that works in real-life situations which contain a variety of constraints 
and necessitate sudden changes in strategy. Examples of these constraints include 
limited time available, delays, and unexpected problems. And second, needs 
assessment training must also equip assessors with other essential skills. Essential 
skills described in the present study include time management, knowledge of change 
agentry, and bias control in data collection. 

Many of the recommendations listed in Table 7 have been incorporated into the 
training situation with positive results. In the year following the study, adjustments 
were made so that the espoused theory of needs assessment more accurately prepared 
novice assessors for conducting one in an actual situation. Components also were 
added that required more rigorous examination of assessors' theories-in-use of needs 
assessment. Changes were made in the needs assessment handout, workshop content, 
and assignment. Examples of changes included: 1) stressing the need to acknowledge 
that assessors had already begun to think about the situation and were likely to have a 
problem and preferred solution in mind, which they needed to document and control, 2) 
warnings to expect time delays and tips on how previous trainees had handled time 
constraints; 3) a change in workshop exercises from trainees receiving input on their 
need assessment plans from staff, to that of peers listening and advising each other 
with staff listening in: and 4) changes in the assignment criteria to specifically require 
detailed reporting of data collection and analysis methods. 
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Table 7 

General Recommendations for the Teaching and Practice of 





Needs Assessment 


Finding # 


Recommendation 


2 


Take a bio k of time to plan NA ( then can do flexibly in small pieces as time 
allows.— F. ruling #11) 


3,4, 5.6, 7 


Cheek for biases between sources and data collection methods 

Seek diversity in sources as to: similarity to assessor's role and both inside and 
out of immediate context 


8, 16 


Acknowledge concept may be difficult, will Icam by doing 



Point or L that NA is not that different from everyday problem solving/dccision 



9 


**■ Prepare novices for time demands, to expect and plan for dclav 


10 


Check motivation level, too high, too invested, too low, don't care (be aware of 
how motivation level influences NA) 


11 


^ Teach key skills along with NA: time management, change ageniry, data 
collection, and analysis bias controls 


12 


Teach how to start wide and gradually focus (for those who have trouble 
narrowing) 

*** Urge to periodically share needs assessment discoveries with a person from outside 
the context (thev will broaden focus) 


13 


Create readily available resources that can be referred to in stages (Handouts work 
well, but also need a person to consult not just while learning about NA, but as 
doing own. This could possibly be same as person sueecsicd just above.) 


14,15 


Acknowledge organizational constraints and show how to overcome with creative 
thinking (internal control) and flexibility 


17 


Show how contacting multiple sources has two, not just one, benefits/purposes, 
be clear on which vou arc doimi with each source 


18 


Acknowledge NAs may be conducted for different purposes (if S. VIE, not to 
discover problem, but to verify and refine) 

*** Whether S ME or novice, need to document WHY doine NA 


1,19,20 


Acknowledge that assessors have already thought about the problem and its 
solution, and design ways to document this so that it can be controlled (as simple 
as writing down before begin, i.c. document preexisting ideas before beginning) 

r*~ Devise a method to sec where and if changes occur as a result of needs assessment. 
For example in developing curricula: in topic (big), in content, in instruction? 

And how big changes were. Follow through by consulting NA in later 
evaluation. 
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IN CONCLUSION 

Argyris and Schon (1974) hypothesize that there are several problems in 
professional education. First, they mention "incongruities between espoused theories 
and theories-in-use" (p. 174). They present the profession of education as a prime 
example of the theory being taught (an espoused theory) diverging widely from the 
reality of practice (the theory-in-use). Argyris and Schon then go on to say that in order 
to begin to resolve differences between espoused theory and theory-in-use, the theory- 
in-use must be explicitly stated. They say that since people often are not aware of this 
conflict and will state their espoused theories when asked, the only way to develop a 
theory-in-use is by directly observing behavior. This is what the present study 
attempted for the topic of needs assessment. 
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